1. Field of the Invention
This invention relates to processing and retrieving data in a database and, more particularly, to processing and retrieving data in a confidential and anonymous manner.
2. Description of the Related Art
In the wake of Sep. 11, 2001 events, various parties (e.g., corporations, governmental agencies or natural persons) face a common dilemma: how can parties share specific information (e.g., a terrorist watch list, black list or a list of actual or potential problematic entities) that would assist the same or separate parties in detecting the presence of potential terrorists or other problematic parties, while maintaining the security and confidentiality of such information and separating any information that is not relevant to the matter?
Hesitation to contribute or otherwise disclose, as well as laws governing the use and disclosure of, certain information is predicated upon a concern that the information could be used in a manner that would violate privacy or otherwise cause damage to the party. Such damage could include identity theft, unauthorized direct marketing activities, unauthorized or intrusive governmental activities, protected class (e.g., racial, religious, gender, ethnic) profiling and discrimination, anti-competitive practices, defamation and/or credit or economic damage.
In response to this dilemma, or any situation requiring the sharing of confidential data, it would be beneficial to have a system wherein various parties may contribute data to an internal or external process or repository in a manner that: (a) sufficiently identifies each record in the data (e.g., a source and record number) without disclosing any entity-identifiable data (e.g., name or social security number); (b) prepares the data so: (i) identical and similar values can be compared regardless of source and (ii) such data can be transmitted in a standard, but confidential, format to protect the confidentiality and security of the data, (c) compares the data to previously contributed data while the data is still in the confidential format, (d) constructs an identifiable entity (such as by utilizing a persistent key and analyzing and enhancing the record with confidential representations of potential aliases, addresses, numbers and/or other identifying information) through matching of the compared data, (e) constructs related entities through an association of the compared data, and/or (f) generates messages for appropriate parties (such as with relevant record identifying elements, e.g., a source and record number), such messages sometimes sent in a confidential manner, such as: (i) on an interval basis and/or wherein at least one message is noise (e.g., a message that does not correspond with a match or relation, but is issued to minimize certain vulnerabilities corresponding with traffic pattern analysis) and (ii) after such message has been processed through a reversible cryptographic algorithm (e.g., encoding, encryption or other algorithm used to engender a level of confidentiality, but can be reversed, such as by using decoding or decryption). Notably, the decrypted message would contain the source and record number, not the underlying protected values.
Current systems use various means to transfer data in a relatively confidential manner within or between parties. For example, some current systems use a reversible cryptographic algorithm, which modifies the data in order to engender some level of confidentiality and lower the risk of losing data during transmission, prior to transmitting data with the understanding that the recipient will use a comparable decoding or decryption method (i.e., an algorithm that reverses, returns or modifies the encoded/encrypted data to a format representative of the original data) in order to decipher and understand the data. However, once the data is deciphered, such data is subject to analysis and use in a manner that could cause the very damage that the encoding/encryption process was intended to protect against.
Other current systems use irreversible cryptographic algorithms (e.g., a one-way function, such as MD-5 or other algorithm used to engender a level of confidentiality, but is irreversible) often as a document signature to make unauthorized document alteration detectable when the document is being shared across parties. Indeed, several existing irreversible cryptographic algorithms cause data to: (a) result in an identical unique value for the same data regardless of source and (b) be undecipherable and irreversible to protect the confidentiality and security of the data. Any minor alteration (such as an extra space) in the data results in a different value after the use of the irreversible cryptographic algorithm as compared with data that does not have the minor alteration, even if the data is otherwise the same. Some current systems utilize an irreversible cryptographic algorithm to process a portion of the data and then match and merge records on a one-to-one basis based upon identical processed data. For example, current systems in a hospital may process the social security numbers in electronic patient records through a one-way function and then match and merge records on a one-to-one basis in a database based upon the processed social security numbers.
However, there are no existing systems that, at a minimum: (a) match received data—after at least a portion of such received data is processed through a cryptographic algorithm (e.g., reversible cryptographic algorithm, such as encoding or decryption, or an irreversible cryptographic algorithm, such as a one-way function) with data previously stored in a database on a one-to-many or many-to-many basis (i.e., received data consists of one or more records that matches data previously stored in a database, the matched data previously stored in a database comprising more than one previously received record), limiting the ability in current systems to build upon identifiable entities while the data is still in a confidential format, (b) move beyond the initial match process to analyze whether any additional information is gained in the initial match and then match other data previously stored in a database based upon the additional information, further limiting the current systems ability to construct identifiable entities, (c) utilizing all or part of those functions identified in (a) and (b) in this paragraph, to match not only same entities, but associate various entities that are determined to be related in some manner (e.g., a passenger on an airline reservation list is a roommate of a natural person on an airline watch list) and/or (d) issue a plurality of messages wherein at least one of the plurality of messages is merely noise.
As such and in addition, there are no existing systems that can use the cryptographic algorithm to share and compare confidential data (including, without limitation, by leaving personally identifiable information in a cryptographic format), construct identifiable or related entities and message the appropriate entities in a manner that maintains security and confidentiality of the original data.
The present invention is provided to address these and other issues. Specifically, the present invention evaluates similarities via a synonym table post-encryption, whereas prior best practices involved the a priori generation of variants and included such variants in the original record.